AppL No. 09/280,421 

Amdt. dated October 18, 2005 

Reply to Office action of May 17, 2005 

Amendments to the Claims: 

This listing of claims will replace all prior versions, and listings, of claims in the application: 
Listing of Claims: 

1 (amended). A method of associating additional information with a video including a plurality 
of frames comprising: 

(a) identifying at least one of said frames representative of a picture composed of a 
plurality of pixels; 

(b) providing a descriptive stream separate from said video; 

(c) including said additional information in said descriptive stream related to said at 
least one of said frames; 

(d) providing said video for displaying on a display; [[and]] 

(e) selectively providing said additional information to a viewer approximately the 
time of said providing said video where said additional information is an object 
depicted by said picture by said pixels[[:]]; 

ffl wherein said descriptive stream includes copyright information indicating the 
owner of the copyright of said video. 

2(original). The method of claim 1 wherein said additional information includes at least one 
of an object index, a textual description, a voice annotation, an image feature, an object link, a URL link, 
and a Java applet. 

3(original). The method of claim 1 wherein said identifying is an object within said frame. 

4(original). The method of claim 1 where said descriptive stream is related to a plurality of 
said frames. 

5(original). The method of claim 4 wherein said at least one of said frames are in sequential 
order in said video. 

6(original). The method of claim 4 wherein said at least one of said frames are in 
nonsequential order in said video. 
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7(original). A method of claim 3 wherein said additional information is related to said object. 

8(original). The method of claim 1 wherein said descriptive stream includes an index 
synchronizing said video with said descriptive stream. 

9(canceled). 

lO(original). The method of claim 1 wherein said descriptive stream is encoded separately 
from said video. 

1 1 (original). The method of claim 10 wherein said video is decoded in the same manner 
independently of whether said descriptive stream is provided. 

12(original). The method of claim 1 1 wherein said video is at least one of MPEG-2 and 
television broadcast format. 

13(original). The method of claim 1 wherein said additional information is presented to said 
viewer on a remote control. 

14(original). The method of claim 1 wherein an audible signal indicates the availability of said 
additional information. 

15(original). The method of claim 1 wherein a visual signal indicates the availability of said 
additional information. 

16(original). The method of claim 7 wherein said additional information includes textual 
based information related to said object. 

17(original). The method of claim 7 wherein said additional information includes audible 
information related to said object. 
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1 8(original). The method of claim 7 wherein said additional information includes image 
features comprising at least one of texture, shape, dominant color, and a motion model related to said 
object. 

19(original). The method of claim 7 wherein said additional information includes links to at 
least one of other objects and frames within said video. 

20(original). The method of claim 7 wherein said additional information includes program 
instructions related to said object. 



21 (amended). A video system comprising: 

(a) an encoder that includes additional information within a video stream including a 
video including a plurality of frames representative of a picture composed of a 
plurality of pixels, where said additional information is related to at least one of 
said frames; 

(b) a receiver that receives said video and said additional information, and said 
receiver decodes said video in the same manner independently of whether said 
additional information is provided; 

(c) a display for displaying said video; [[and]] 

(d) a trigger mechanism for selectively presenting said additional information to a 
viewer at approximately the time of presenting said frames to said viewer where 
said additional information is an object depicted by said picture by said 
pixels[[.]]; 

(e) wherein said descriptive stream includes copyright information indicating the 
owner of the copyright of said video. 

22(original). The system of claim 21, further comprising: 

(a) a transmitter for transmitting said video signal and said additional information; 
and 

(b) a receiver for receiving said video signal and said additional information. 



23(original). 
and a computer. 



The system of claim 22 wherein said encoder is at least one of a video camera 
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24(original). The system of claim 21 wherein said trigger mechanism is located in a remote 
control device. 

25(original). The system of claim 21 wherein said additional information is provided by a 
remote control device. 

26(original). The method of claim 21 wherein said additional information is related to an 
object within said frame and includes links to at least one of other objects and frames within said video. 

27(original). The method of claim 21 wherein said additional information is related to an 
object within said frame and includes program instructions related to said object. 

28(original). The method of claim 21 wherein said additional information is related to an 
object within said frame and includes textual based information related to said object. 

29(original). The method of claim 21 wherein said additional information is related to an 
object within said frame and includes audible information related to said object. 

30(original). The method of claim 21 wherein said additional information is related to an 
object within said frame and includes image features comprising at least one of texture, shape, dominant 
color, and a motion model related to said object. 

3 1 (amended). A system for presenting information comprising: 

(a) a unitary file stored on a recordable media containing both [[an]] only a single 
image representative of a picture composed of a plurality of pixels and additional 
information associated with said image , wherein said image is stored in a first 
part of said file and said additional information is stored in a second part of said 

fik; 

(b) a selection mechanism that permits the selection of graphical objects depicted by 
said picture by said pixels in said image for which said additional information is 
related thereto; and 
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(c) a presentation mechanism that provides said additional information to a viewer in 
response to selecting said object depicted by said picture by said pixels. 

32(original). The system of claim 3 1 wherein said file includes said image followed by said 
additional information. 

33(original). The system of claim 32 wherein said image and said additional information are 
separated by a marker indicating the end of said image. 

34(original). The system of claim 33 wherein an image viewer which does not recognize said 
additional information will display said image properly and recognize said marker as indicating the end of 
said image. 

35(original). The system of claim 34 wherein said image is in a JPEG format. 

36(original). The system of claim 3 1 wherein said additional information is organized in at 
least two layers comprising: 

(a) a first layer containing information describing the location of objects within said 
image; and 

(b) a second layer containing additional information regarding said objects within 
said image, where said first layer contains fewer bytes than said second layer. 

37(original). The system of claim 36 wherein said second layer follows said first layer, which 
in turn follows said image file. 

38(original). The system of claim 36 wherein said first layer contains a length identifier 
describing the length of said first layer. 

39(original). The system of claim 36 wherein said first layer contains a number of objects 
identifier describing the number of objects identified by said first layer. 

40(original). The system of claim 36 wherein said first layer contains a number of data 
identifier describing the number of data items associated with a particular said object. 
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41 (original). The system of claim 36 wherein said first layer contains a first definition of the 
outline of an object of said image. 

42(original). The system of claim 36 wherein said second layer contains a length identifier 
describing the length of said second layer. 

43(original). The system of claim 36 wherein said second layer contains an array of offsets 
that identify the start of each data item. 

44(original). The system of claim 41 wherein said second layer contains a second definition of 
the outline of said object of said image, where said second definition more closely approximates the 
outline of said object than said first definition. 

45(original). The system of claim 41 wherein said second layer contains a second definition of 
the outline of said object of said image, where said second definition contains more bytes than said first 
definition. 



46(original). The system of claim 36 wherein said second layer includes sound data related to 
said object. 

47(original). The system of claim 36 wherein said second layer includes HTML meta tags 
related to said object. 

48(original). The system of claim 36 wherein said second layer includes textual annotations 
related to said object. 

49(original). The system of claim 36 wherein said second layer includes an HTML page to be 
rendered. 



50(original). The system of claim 36 wherein said second layer includes a Java Applet related 
to said object. 
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5 1 (original). The system of claim 36 wherein said second layer includes a color histogram. 

52(original). The system of claim 36 wherein said second layer includes data related to the 
conditions under which said image was created including at least one of lighting, camera settings, and 
time of acquisition. 

53(original). The system of claim 36 wherein said second layer includes data related to 
information for reproducing said image including at least one of cropping information, paper type, camera 
settings, and image production settings. 

54(original). The system of claim 36 wherein said second layer includes another image to be 
superimposed upon said image. 

55(original). The system of claim 36 wherein said second layer includes data regarding the 
author of said image. 

56(original). The system of claim 36 wherein said second layer includes copyright data 
regarding the copyright of said image. 

57(original). The system of claim 56 said copyright data is encoded. 

58(original). The system of claim 36 wherein said second layer includes information regarding 
how said image should be viewed. 

59(amended). A method of presenting information comprising: 

(a) displaying an image from a unitary file stored on a recordable media containing 
both [[an]] only a single image representative of a picture composed of a 
plurality of pixels and additional information associated with said image , where 
said image is stored in a first part of said file and said additional information is 
stored in a second part of said file : 

(b) selecting an object depicted by said picture by said pixels in said image for which 
said additional information is related thereto; and 
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(c) providing said additional information to a viewer in response to selecting said 
object depicted by said picture by said pixels. 

60(original). The method of claim 59 wherein said file includes said image followed by said 
additional information. 

61 (original). The method of claim 60 wherein said image and said additional information are 
separated by a marker indicating the end of said image. 

62(original). The method of claim 61 wherein an image viewer which does not recognize said 
additional information will display said image properly and recognize said marker as indicating the end of 
said image. 

63(original). The method of claim 62 wherein said image is in a JPEG format. 

64(original). The method of claim 59 wherein said additional information is organized in at 
least two layers comprising a first layer containing information describing the location of objects within 
said image and a second layer containing additional information regarding said objects within said image, 
where said first layer contains fewer bytes than said second layer. 

65(original). The method of claim 64 wherein said second layer follows said first layer, which 
in turn follows said image file. 

66(original). The method of claim 64 wherein said first layer contains a length identifier 
describing the length of said first layer. 

67(original). The method of claim 64 wherein said first layer contains a number of objects 
identifier describing the number of objects identified by said first layer. 

68(original). The method of claim 64 wherein said first layer contains a number of data 
identifier describing the number of data items associated with a particular said object. 
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69(original). The method of claim 64 wherein said first layer contains a first definition of the 
outline of an object of said image. 

70(original). The method of claim 64 wherein said second layer contains a length identifier 
describing the length of said second layer. 

71 (original). The method of claim 64 wherein said second layer contains an array of offsets 
that identify the start of each data item. 

72(original). The method of claim 69 wherein said second layer contains a second definition 
of the outline of said object of said image, where said second definition more closely approximates the 
outline of said object than said first definition. 

73(original). The method of claim 69 wherein said second layer contains a second definition 
of the outline of said object of said image, where said second definition contains more bytes than said first 
definition. 

74(original). The method of claim 64 wherein said second layer includes sound data related to 
said object. 

75(original). The method of claim 64 wherein said second layer includes HTML meta tags 
related to said object. 

76(original). The method of claim 64 wherein said second layer includes textual annotations 
related to said object. 

77(original). The method of claim 64 wherein said second layer includes an HTML page to be 
rendered. 

78(original). The method of claim 64 wherein said second layer includes a Java Applet related 
to said object. 

79(original). The method of claim 64 wherein said second layer includes a color histogram. 
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80(original). The method of claim 64 wherein said second layer includes data related to the 
conditions under which said image was created including at least one of lighting, camera settings, and 
time of acquisition. 

81 (original). The method of claim 64 wherein said second layer includes data related to 
information for reproducing said image including at least one of cropping information, paper type, camera 
settings, and image production settings. 

82(original). The method of claim 64 wherein said second layer includes another image to be 
superimposed upon said image. 

83(original). The method of claim 64 wherein said second layer includes data regarding the 
author of said image. 



84(original). The method of claim 64 wherein said second layer includes copyright data 
regarding the copyright of said image. 

85(original). The method of claim 64 said copyright data is encoded. 

86(original). The method of claim 64 wherein said second layer includes information 
regarding how said image should be viewed. 



87(amended ). A method for creating an image file comprising: 

(a) providing [[an]] a single image representative of a picture composed of a 
plurality of pixels; 

(b) selecting an object depicted by said picture by said pixels in said image; 

(c) providing additional information related to said object; and 

(d) storing both said image and said additional information in a unitary file stored 
on a recordable media, where said file includes only a single image and 
said image is stored in a first part of said file and said additional 
information is stored in a second part of said file . 
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88(original). The method of claim 87 wherein said file includes said image followed by said 
additional information. 

89(original). The method of claim 88 wherein said image and said additional information are 
separated by a marker indicating the end of said image. 

90(original). The method of claim 89 wherein an image viewer which does not recognize said 
additional information will display said image properly and recognize said marker as indicating the end of 
said image. 

91 (original). The method of claim 90 wherein said image is in a JPEG format. 

92(original). The method of claim 87 wherein said additional information is organized in at 
least two layers comprising a first layer containing information describing the location of objects within 
said image and a second layer containing additional information regarding said objects within said image, 
where said first layer contains fewer bytes than said second layer. 

93(original). The method of claim 92 wherein said second layer follows said first layer, which 
in turn follows said image file. 

94(original). The method of claim 92 wherein said first layer contains a length identifier 
describing the length of said first layer. 

95(original). The method of claim 92 wherein said first layer contains a number of objects 
identifier describing the number of objects identified by said first layer. 

96(original). The method of claim 92 wherein said first layer contains a number of data 
identifier describing the number of data items associated with a particular said object. 

97(original). The method of claim 92 wherein said first layer contains a first definition of the 
outline of an object of said image. 
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98(original). The method of claim 92 wherein said second layer contains a length identifier 
describing the length of said second layer. 

99(original). The method of claim 92 wherein said second layer contains an array of offsets 
that identify the start of each data item. 

lOO(original). The method of claim 97 wherein said second layer contains a second definition 
of the outline of said object of said image, where said second definition more closely approximates the 
outline of said object than said first definition. 

lOl(original). The method of claim 97 wherein said second layer contains a second definition 
of the outline of said object of said image, where said second definition contains more bytes than said first 
definition. 



102(original). The method of claim 92 wherein said second layer includes sound data related to 
said object. 

103(original). The method of claim 92 wherein said second layer includes HTML meta tags 
related to said object. 

104(original). The method of claim 92 wherein said second layer includes textual annotations 
related to said object. 

105(original). The method of claim 92 wherein said second layer includes an HTML page to be 
rendered. 



106(original). The method of claim 92 wherein said second layer includes a Java Applet related 
to said object. 

107(original). The method of claim 92 wherein said second layer includes a color histogram. 
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108(original). The method of claim 92 wherein said second layer includes data related to the 
conditions under which said image was created including at least one of lighting, camera settings, and 
time of acquisition. 

109(original). The method of claim 92 wherein said second layer includes data related to 
information for reproducing said image including at least one of cropping information, paper type, camera 
settings, and image production settings. 

1 lO(original). The method of claim 92 wherein said second layer includes another image to be 
superimposed upon said image. 

1 1 1 (original). The method of claim 92 wherein said second layer includes data regarding the 
author of said image. 

1 12(original). The method of claim 92 wherein said second layer includes copyright data 
regarding the copyright of said image. 

1 13(original). The method of claim 92 said copyright data is encoded. 

1 14(original). The method of claim 92 wherein said second layer includes information regardin 
how said image should be viewed. 

1 15(original). The method of claim 92 wherein said first layer is transmitted from a first 
computer to a second computer together with said image. 

1 1 6(original). The method of claim 1 1 5 wherein portions of said second layer are transmitted 
from said first computer to said second computer upon request by said first computer. 

1 17(original). The method of claim 1 16 wherein said request is in response to a user selecting 
an object within said image. 
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